Session 3: Human Language Evaluation
نویسنده
چکیده
* Cross-system evaluation: This is a mainstay of the periodic ARPA evaluations on competing systems. Multiple sites agree to run their respective systems on a single application, so that results across systems are comparable. This includes evaluations such as message understanding (MUC)[6], information retrieval (TREC)[7], spoken language systems (ATIS)[8], and automated speech recognition (CSR)[8].
منابع مشابه
The Correlation of Machine Translation Evaluation Metrics with Human Judgement on Persian Language
Machine Translation Evaluation Metrics (MTEMs) are the central core of Machine Translation (MT) engines as they are developed based on frequent evaluation. Although MTEMs are widespread today, their validity and quality for many languages is still under question. The aim of this research study was to examine the validity and assess the quality of MTEMs from Lexical Similarity set on machine tra...
متن کاملSession 3: Natural Language Evaluation
The session on Natural Language Evaluation focused on methods for evaluating text understanding systems. Beginning with the first Message Understanding Conference (MUCK-l) in 1987, there has been increasing focus on how to measure and evaluate text understanding systems. The MUCK-1 conference required developers to port their system to a common domain of Navy intelligence messages; MUCK-2 (May ...
متن کاملPITT at TREC 2011 Session Track
In this paper, we introduce our approaches for TREC 2011 session track. Our approaches focus on combining different query language models to model information needs in a search session. In RL1 stage, we build ad hoc retrieval system using sequential dependence model (SDM) on current query. In RL2 stage, we build query language models by combining SDM features (e.g. single term, ordered phrase, ...
متن کاملSession 10: Corpora and Evaluation
This session on corpora and evaluation was composed of two distinct parts. Before the break, four papers dealing with a range of important aspects of evaluation of written language systems and spoken language systems were presented. A printed version of each of these papers is included in the conference proceedings. After the break, a series of informal reports (not included as proceedings pape...
متن کاملSession 1: Lexicons, Corpora, and Evaluation
Our technologies for collecting, storing, and disseminating vast amounts of information have gotten ahead of our technologies for collating and analyzing it, and that situation has posed a serious challenge for human language technology. As a consequence, natural language processing has been moving rapidly towards large-scale systems addressed to real tasks. Demos that won't scale up are no lon...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 1994